Nonuniqueness versus Uniqueness of Optimal Policies in Convex Discounted Markov Decision Processes

Uniqueness of optimal policies as a generic property of discounted Markov decision processes: Ekeland's variational principle approach

Accelerated decomposition techniques for large discounted Markov decision processes

Many hierarchical techniques to solve large Markov decision processes (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorith...

A Robust Constrained Markov Decision Process Model for Admission Control in a Single Server Queue

This paper presents a robust optimization approach for discounted constrained Markov decision processes with payoff uncertainty. It is assumed that the decision-maker has no distributional information on the unknown payoffs. Two types of uncertainty sets, convex hulls and intervals are considered. Interval uncertainty sets are parametrized allowing a subset of the payoffs to vary within interva...

On the Convergence of Optimal Actions for Markov Decision Processes and the Optimality of (s, S) Inventory Policies

This paper studies convergence properties of optimal values and actions for discounted and averagecost Markov Decision Processes (MDPs) with weakly continuous transition probabilities and applies these properties to the stochastic periodic-review inventory control problem with backorders, positive setup costs, and convex holding/backordering costs. The following results are established for MDPs...

Total Expected Discounted Reward MDPs: Existence of Optimal Policies

This article describes the results on the existence of optimal and nearly optimal policies for Markov Decision Processes (MDPs) with total expected discounted rewards. The problem of optimization of total expected discounted rewards for MDPs is also known under the name of discounted dynamic programming.